{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced Graphs\n",
    "\n",
    "Prequisites:\n",
    "\n",
    " - A running Kubernetes cluster\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - [seldon-core Python package](https://pypi.org/project/seldon-core/) (```pip install seldon-core```)\n",
    " - [Helm](https://github.com/kubernetes/helm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we will illustrate the different types of microservices that can be deployed in Seldon:\n",
    "* Model\n",
    "* Transformer\n",
    "* Router\n",
    "* Combiner\n",
    "* Output Transformer\n",
    "\n",
    "We will deploy graphs of increasing complexity. But first we need to install seldon on the cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If running via Minikube you may need to ensure you have cluster-admin rights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created']"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "!!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount/tiller created\n",
      "clusterrolebinding.rbac.authorization.k8s.io/tiller created\n",
      "$HELM_HOME has been configured at /home/janis/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core-crd\n",
      "LAST DEPLOYED: Tue Dec 18 13:55:49 2018\n",
      "NAMESPACE: default\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-spartakus-config  3     1s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  1s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  1        0        0           0          0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                        SECRETS  AGE\n",
      "seldon-spartakus-volunteer  1        0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core-crd --name seldon-core-crd --set usage_metrics.enabled=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/graphs created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Tue Dec 18 13:55:58 2018\n",
      "NAMESPACE: graphs\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/Service\n",
      "NAME                          TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)                        AGE\n",
      "seldon-core-seldon-apiserver  NodePort   10.110.96.213  <none>       8080:30658/TCP,5000:30103/TCP  1s\n",
      "seldon-core-redis             ClusterIP  10.99.138.221  <none>       6379/TCP                       1s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                                DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-core-seldon-apiserver        1        1        1           0          1s\n",
      "seldon-core-seldon-cluster-manager  1        1        1           0          1s\n",
      "seldon-core-redis                   1        1        1           0          0s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                                 READY  STATUS             RESTARTS  AGE\n",
      "seldon-core-seldon-apiserver-59978fbf45-5xvf5        0/1    ContainerCreating  0         0s\n",
      "seldon-core-seldon-cluster-manager-5449b8dd5f-9kfvp  0/1    ContainerCreating  0         0s\n",
      "seldon-core-redis-567db9ddcf-h88st                   0/1    ContainerCreating  0         0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME    SECRETS  AGE\n",
      "seldon  1        1s\n",
      "\n",
      "==> v1beta1/Role\n",
      "NAME          AGE\n",
      "seldon-local  1s\n",
      "\n",
      "==> v1/RoleBinding\n",
      "NAME    AGE\n",
      "seldon  1s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Thank you for installing Seldon Core.\n",
      "\n",
      "Documentation can be found at https://github.com/SeldonIO/seldon-core\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core --name seldon-core \\\n",
    "        --set cluster_manager.rbac=true \\\n",
    "        --namespace graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up REST and gRPC methods\n",
    "\n",
    "**Ensure you port forward the seldon api-server REST and GRPC ports**:\n",
    "\n",
    "REST:\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n graphs -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n graphs 8002:8080\n",
    "```\n",
    "\n",
    "GRPC:\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n graphs -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n graphs 8003:5000\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from visualizer import get_graph\n",
    "import json\n",
    "from seldon_utils import *\n",
    "API_GATEWAY_REST=\"localhost:8002\"\n",
    "API_GATEWAY_GRPC=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"122pt\" height=\"171pt\"\n",
       " viewBox=\"0.00 0.00 122.00 171.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 167)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-167 118,-167 118,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\">\n",
       "<title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"8,-8 8,-155 106,-155 106,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"57\" y=\"-139.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predictor&#45;0</text>\n",
       "</g>\n",
       "<!-- classifier0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>classifier0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"98,-124 16,-124 16,-88 98,-88 98,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"57\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier</text>\n",
       "</g>\n",
       "<!-- classifier0endpoint -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>classifier0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"57\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"57\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier0&#45;&gt;classifier0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>classifier0&#45;&gt;classifier0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M57,-87.8314C57,-80.131 57,-70.9743 57,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"60.5001,-62.4132 57,-52.4133 53.5001,-62.4133 60.5001,-62.4132\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f4c169c3860>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/model.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we will check that everything works by running a simple model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-model created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/model.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-example-7cd068f replicas:1 replicasAvailable:1]] state:Available]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-model -o jsonpath=\"{.status}\" -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":43199,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"lkkh68f0t6bg0ue0utfsuo70pu\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.10111662639415119]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":43198,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"u90pon2u2gqj1oku0cpou7psci\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.07163270376831207\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"seldon-model\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Random AB Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"249pt\" height=\"243pt\"\n",
       " viewBox=\"0.00 0.00 249.00 243.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 239)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-239 245,-239 245,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\">\n",
       "<title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"8,-8 8,-227 233,-227 233,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"120.5\" y=\"-211.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predictor&#45;0</text>\n",
       "</g>\n",
       "<!-- random&#45;ab&#45;test0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>random&#45;ab&#45;test0</title>\n",
       "<polygon fill=\"#d3d3d3\" stroke=\"#d3d3d3\" points=\"182.5,-196 57.5,-196 57.5,-160 182.5,-160 182.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"120\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">random&#45;ab&#45;test</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"111.5,-124 16.5,-124 16.5,-88 111.5,-88 111.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"64\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- random&#45;ab&#45;test0&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>random&#45;ab&#45;test0&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M105.8688,-159.8314C99.4189,-151.5386 91.6555,-141.557 84.5748,-132.4533\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"87.2236,-130.158 78.3214,-124.4133 81.6981,-134.4556 87.2236,-130.158\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"224.5,-124 129.5,-124 129.5,-88 224.5,-88 224.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"177\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- random&#45;ab&#45;test0&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>random&#45;ab&#45;test0&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M134.3835,-159.8314C140.9486,-151.5386 148.8507,-141.557 156.0578,-132.4533\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"158.9599,-134.4262 162.4228,-124.4133 153.4716,-130.0813 158.9599,-134.4262\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"64\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"64\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M64,-87.8314C64,-80.131 64,-70.9743 64,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"67.5001,-62.4132 64,-52.4133 60.5001,-62.4133 67.5001,-62.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"177\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"177\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M177,-87.8314C177,-80.131 177,-70.9743 177,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"180.5001,-62.4132 177,-52.4133 173.5001,-62.4133 180.5001,-62.4132\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f4c16763f28>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/random_ab_test.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we will deploy 2 models under an AB test router. The Random AB Test we will use is implemented directly in Seldon, this is not a microservice, so no docker image needs to be specified.\n",
    "\n",
    "The json graph is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'random-ab-test',\n",
       " 'endpoint': {},\n",
       " 'implementation': 'RANDOM_ABTEST',\n",
       " 'parameters': [{'name': 'ratioA', 'value': '0.5', 'type': 'FLOAT'}],\n",
       " 'children': [{'name': 'classifier-1',\n",
       "   'endpoint': {'type': 'REST'},\n",
       "   'type': 'MODEL',\n",
       "   'children': []},\n",
       "  {'name': 'classifier-2',\n",
       "   'endpoint': {'type': 'REST'},\n",
       "   'type': 'MODEL',\n",
       "   'children': []}]}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/random_ab_test.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We specify ``` \"implementation\": \"RANDOM_ABTEST\" ``` to get the AB Test router implemented in Seldon.\n",
    "We pass the parameter ratioA which corresponds to the ratio of requests that will be passed to the first child of the Router"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-deployment created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/random_ab_test.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[state:Available predictorStatus:[map[name:test-deployment-abtest-abtest-41de5b8 replicas:1 replicasAvailable:1] map[replicasAvailable:1 name:test-deployment-abtest-abtest-df66c5c replicas:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":43112,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"9ij7bk9sg0rsgon7ja7aosd3i9\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"random-ab-test\": 1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"random-ab-test\": \"\",\n",
      "      \"classifier-2\": \"seldonio/mock_classifier:1.0\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.0685356300966698]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":43110,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"ijlor60vst31vobptbld5q25lt\"\n",
       "  routing {\n",
       "    key: \"random-ab-test\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"random-ab-test\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.08929570089111957\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/random_ab_test.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Average Combiner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"362pt\" height=\"243pt\"\n",
       " viewBox=\"0.00 0.00 362.00 243.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 239)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-239 358,-239 358,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\">\n",
       "<title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"8,-8 8,-227 346,-227 346,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"177\" y=\"-211.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predictor&#45;0</text>\n",
       "</g>\n",
       "<!-- ensemble0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>ensemble0</title>\n",
       "<polygon fill=\"#d3d3d3\" stroke=\"#d3d3d3\" points=\"219.5,-196 134.5,-196 134.5,-160 219.5,-160 219.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"177\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">ensemble</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"111.5,-124 16.5,-124 16.5,-88 111.5,-88 111.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"64\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- ensemble0&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>ensemble0&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M148.4853,-159.8314C134.1658,-150.7074 116.6376,-139.539 101.2662,-129.7449\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"102.8199,-126.5847 92.5055,-124.1628 99.0583,-132.4882 102.8199,-126.5847\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"224.5,-124 129.5,-124 129.5,-88 224.5,-88 224.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"177\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- ensemble0&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>ensemble0&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M177,-159.8314C177,-152.131 177,-142.9743 177,-134.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"180.5001,-134.4132 177,-124.4133 173.5001,-134.4133 180.5001,-134.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>classifier&#45;3</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"337.5,-124 242.5,-124 242.5,-88 337.5,-88 337.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"290\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;3</text>\n",
       "</g>\n",
       "<!-- ensemble0&#45;&gt;classifier&#45;3 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>ensemble0&#45;&gt;classifier&#45;3</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M205.5147,-159.8314C219.8342,-150.7074 237.3624,-139.539 252.7338,-129.7449\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"254.9417,-132.4882 261.4945,-124.1628 251.1801,-126.5847 254.9417,-132.4882\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"64\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"64\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M64,-87.8314C64,-80.131 64,-70.9743 64,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"67.5001,-62.4132 64,-52.4133 60.5001,-62.4133 67.5001,-62.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"177\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"177\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M177,-87.8314C177,-80.131 177,-70.9743 177,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"180.5001,-62.4132 177,-52.4133 173.5001,-62.4133 180.5001,-62.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3endpoint -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>classifier&#45;3endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"290\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"290\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;3&#45;&gt;classifier&#45;3endpoint -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>classifier&#45;3&#45;&gt;classifier&#45;3endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M290,-87.8314C290,-80.131 290,-70.9743 290,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"293.5001,-62.4132 290,-52.4133 286.5001,-62.4133 293.5001,-62.4132\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f4c1678be80>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/ensemble.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example again we will use a service implemented in seldon, called Average Combiner. It takes the outputs of several models and returns the arithmetic mean of them.\n",
    "\n",
    "The json is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'ensemble',\n",
       " 'implementation': 'AVERAGE_COMBINER',\n",
       " 'children': [{'name': 'classifier-1',\n",
       "   'endpoint': {'type': 'REST'},\n",
       "   'type': 'MODEL'},\n",
       "  {'name': 'classifier-2', 'endpoint': {'type': 'REST'}, 'type': 'MODEL'},\n",
       "  {'name': 'classifier-3', 'endpoint': {'type': 'REST'}, 'type': 'MODEL'}]}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/ensemble.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-deployment created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/ensemble.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-ensembles-ensemble-2f95e01 replicas:1 replicasAvailable:1]] state:Available]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":43030,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"status\": {\n",
      "    \"code\": 0,\n",
      "    \"info\": \"\",\n",
      "    \"reason\": \"\",\n",
      "    \"status\": \"SUCCESS\"\n",
      "  },\n",
      "  \"meta\": {\n",
      "    \"puid\": \"6go0ibhhdrqdiitm5ij8riu40r\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"ensemble\": -1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"ensemble\": \"\",\n",
      "      \"classifier-3\": \"seldonio/mock_classifier:1.0\",\n",
      "      \"classifier-2\": \"seldonio/mock_classifier:1.0\",\n",
      "      \"classifier-1\": \"seldonio/mock_classifier:1.0\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.09152959251363695]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":43029,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "status {\n",
       "}\n",
       "meta {\n",
       "  puid: \"meltdm6n2pnvpq5mqfbr3fjh5r\"\n",
       "  routing {\n",
       "    key: \"ensemble\"\n",
       "    value: -1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-2\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-3\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"ensemble\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.06965596912580471\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/ensemble.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"214pt\" height=\"243pt\"\n",
       " viewBox=\"0.00 0.00 214.00 243.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 239)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-239 210,-239 210,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\">\n",
       "<title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"8,-8 8,-227 198,-227 198,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"103\" y=\"-211.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predictor&#45;0</text>\n",
       "</g>\n",
       "<!-- transformer0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>transformer0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"154.5,-196 51.5,-196 51.5,-160 154.5,-160 154.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"103\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">transformer</text>\n",
       "</g>\n",
       "<!-- transformer0endpoint -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>transformer0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"53\" cy=\"-106\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"53\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- transformer0&#45;&gt;transformer0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>transformer0&#45;&gt;transformer0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M90.3829,-159.8314C84.5074,-151.3707 77.4111,-141.1521 70.9871,-131.9014\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"73.6744,-129.635 65.0956,-123.4177 67.9248,-133.6278 73.6744,-129.635\"/>\n",
       "</g>\n",
       "<!-- classifier -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>classifier</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"190,-124 108,-124 108,-88 190,-88 190,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"149\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier</text>\n",
       "</g>\n",
       "<!-- transformer0&#45;&gt;classifier -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>transformer0&#45;&gt;classifier</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M114.6077,-159.8314C119.7978,-151.7079 126.0232,-141.9637 131.7425,-133.0118\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"134.8015,-134.7246 137.236,-124.4133 128.9026,-130.9559 134.8015,-134.7246\"/>\n",
       "</g>\n",
       "<!-- classifierendpoint -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>classifierendpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"149\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"149\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;&gt;classifierendpoint -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>classifier&#45;&gt;classifierendpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M149,-87.8314C149,-80.131 149,-70.9743 149,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"152.5001,-62.4132 149,-52.4133 145.5001,-62.4133 152.5001,-62.4132\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f4c16720828>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/feature_transform.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we deploy a simple model under a feature transformation microservice. For the transformer we will use the docker image seldonio/mock_transformer:1.0\n",
    "\n",
    "Since this is not implemented in Seldon, the type of predictive unit (TRANSFORMER) needs to be specified in the graph so that Seldon Core knows which API this microservice implements.\n",
    "\n",
    "The json is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'transformer',\n",
       " 'endpoint': {'type': 'REST'},\n",
       " 'type': 'TRANSFORMER',\n",
       " 'children': [{'name': 'classifier',\n",
       "   'endpoint': {'type': 'REST'},\n",
       "   'type': 'MODEL'}]}"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/feature_transform.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-deployment created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/feature_transform.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[state:Available predictorStatus:[map[replicasAvailable:1 name:test-deployment-transformer-transformer-c7a49aa replicas:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":42934,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"tocfi25mkk7mrvvadv2qk3kcf9\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"transformer\": -1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"transformer\": \"seldonio/mock_transformer:1.0\",\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07466519122340777]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":42933,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"l23uvl0par42feolqpf9pns7il\"\n",
       "  routing {\n",
       "    key: \"transformer\"\n",
       "    value: -1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"transformer\"\n",
       "    value: \"seldonio/mock_transformer:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.09676272871657236\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/feature_transform.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outlier Detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"254pt\" height=\"315pt\"\n",
       " viewBox=\"0.00 0.00 254.00 315.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 311)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-311 250,-311 250,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\">\n",
       "<title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"8,-8 8,-299 238,-299 238,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"123\" y=\"-283.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predictor&#45;0</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>outlier&#45;detector0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"187.5,-268 58.5,-268 58.5,-232 187.5,-232 187.5,-268\"/>\n",
       "<text text-anchor=\"middle\" x=\"123\" y=\"-246.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">outlier&#45;detector</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0endpoint -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>outlier&#45;detector0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"53\" cy=\"-178\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"53\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M105.336,-231.8314C96.4282,-222.669 85.516,-211.445 75.9651,-201.6213\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"78.4629,-199.1695 68.9826,-194.4393 73.444,-204.049 78.4629,-199.1695\"/>\n",
       "</g>\n",
       "<!-- random&#45;abtest -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>random&#45;abtest</title>\n",
       "<polygon fill=\"#d3d3d3\" stroke=\"#d3d3d3\" points=\"229.5,-196 108.5,-196 108.5,-160 229.5,-160 229.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"169\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">random&#45;abtest</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;random&#45;abtest -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>outlier&#45;detector0&#45;&gt;random&#45;abtest</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M134.6077,-231.8314C139.7978,-223.7079 146.0232,-213.9637 151.7425,-205.0118\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"154.8015,-206.7246 157.236,-196.4133 148.9026,-202.9559 154.8015,-206.7246\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"113.5,-124 18.5,-124 18.5,-88 113.5,-88 113.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"66\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>random&#45;abtest&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M143.0088,-159.8314C130.0784,-150.7927 114.2778,-139.7476 100.3617,-130.0198\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"102.1843,-127.0235 91.9829,-124.1628 98.1737,-132.7608 102.1843,-127.0235\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"226.5,-124 131.5,-124 131.5,-88 226.5,-88 226.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"179\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>random&#45;abtest&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M171.5234,-159.8314C172.5929,-152.131 173.8647,-142.9743 175.0532,-134.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"178.5336,-134.7997 176.4426,-124.4133 171.6001,-133.8367 178.5336,-134.7997\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"66\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"66\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M66,-87.8314C66,-80.131 66,-70.9743 66,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"69.5001,-62.4132 66,-52.4133 62.5001,-62.4133 69.5001,-62.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"179\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"179\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M179,-87.8314C179,-80.131 179,-70.9743 179,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"182.5001,-62.4132 179,-52.4133 175.5001,-62.4133 182.5001,-62.4132\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f4c167634e0>"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/outlier_detector.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we will have different four components: \n",
    "* A transformer, the outlier detector\n",
    "* A router, the random AB test\n",
    "* Two models\n",
    "\n",
    "The outlier detector is a special kind of transformer that will populate a tag in the response metadata with the outlier score it has calculated. \n",
    "We use the docker image seldonio/outlier_mahalanobis:0.2 for the outlier detector.\n",
    "\n",
    "The json is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'outlier-detector',\n",
       " 'type': 'TRANSFORMER',\n",
       " 'endpoint': {'type': 'REST'},\n",
       " 'children': [{'name': 'random-abtest',\n",
       "   'implementation': 'RANDOM_ABTEST',\n",
       "   'type': 'UNKNOWN_TYPE',\n",
       "   'parameters': [{'name': 'ratioA', 'value': '0.5', 'type': 'FLOAT'}],\n",
       "   'children': [{'name': 'classifier-1',\n",
       "     'type': 'MODEL',\n",
       "     'endpoint': {'type': 'REST'}},\n",
       "    {'name': 'classifier-2', 'type': 'MODEL', 'endpoint': {'type': 'REST'}}]}]}"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/outlier_detector.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-deployment created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/outlier_detector.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-complex-outlier-8c0fd6e replicas:1 replicasAvailable:0] map[name:test-deployment-complex-outlier-6180537 replicas:1 replicasAvailable:1]] state:Creating]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":42857,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"lv5ofsaq753b500vqrrjqprb8r\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": [0.0, 0.0]\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1,\n",
      "      \"random-abtest\": 1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"outlier-detector\": \"seldonio/outlier_mahalanobis:0.2\",\n",
      "      \"classifier-2\": \"seldonio/mock_classifier:1.0\",\n",
      "      \"random-abtest\": \"\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [2, 1],\n",
      "      \"values\": [0.07693220932086946, 0.08538117568148401]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST,data_size=10,rows=2)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":42856,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"le74kjvmtkoputduf0bmddthtn\"\n",
       "  tags {\n",
       "    key: \"outlierScore\"\n",
       "    value {\n",
       "      list_value {\n",
       "        values {\n",
       "          number_value: 0.0\n",
       "        }\n",
       "        values {\n",
       "          number_value: 0.0\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  routing {\n",
       "    key: \"outlier-detector\"\n",
       "    value: -1\n",
       "  }\n",
       "  routing {\n",
       "    key: \"random-abtest\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"outlier-detector\"\n",
       "    value: \"seldonio/outlier_mahalanobis:0.2\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"random-abtest\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 2\n",
       "    shape: 1\n",
       "    values: 0.08967091562607983\n",
       "    values: 0.07605855257155911\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST,API_GATEWAY_GRPC,data_size=10,rows=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/outlier_detector.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complex Graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"454pt\" height=\"387pt\"\n",
       " viewBox=\"0.00 0.00 454.00 387.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 383)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-383 450,-383 450,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\">\n",
       "<title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"8,-8 8,-371 438,-371 438,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"223\" y=\"-355.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">predictor&#45;0</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>outlier&#45;detector0</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"189.5,-340 60.5,-340 60.5,-304 189.5,-304 189.5,-340\"/>\n",
       "<text text-anchor=\"middle\" x=\"125\" y=\"-318.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">outlier&#45;detector</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0endpoint -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>outlier&#45;detector0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"89\" cy=\"-250\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"89\" y=\"-246.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M115.9157,-303.8314C111.8344,-295.6688 106.935,-285.87 102.4414,-276.8827\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"105.5594,-275.2924 97.9567,-267.9134 99.2984,-278.423 105.5594,-275.2924\"/>\n",
       "</g>\n",
       "<!-- random&#45;abtest -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>random&#45;abtest</title>\n",
       "<polygon fill=\"#d3d3d3\" stroke=\"#d3d3d3\" points=\"265.5,-268 144.5,-268 144.5,-232 265.5,-232 265.5,-268\"/>\n",
       "<text text-anchor=\"middle\" x=\"205\" y=\"-246.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">random&#45;abtest</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;random&#45;abtest -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>outlier&#45;detector0&#45;&gt;random&#45;abtest</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M145.1874,-303.8314C154.8514,-295.1337 166.5796,-284.5783 177.0816,-275.1265\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"179.7275,-277.454 184.8191,-268.1628 175.0447,-272.251 179.7275,-277.454\"/>\n",
       "</g>\n",
       "<!-- transformer -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>transformer</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"207.5,-196 104.5,-196 104.5,-160 207.5,-160 207.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"156\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">transformer</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;transformer -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>random&#45;abtest&#45;&gt;transformer</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M192.6352,-231.8314C187.0491,-223.6232 180.337,-213.7606 174.1928,-204.7323\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"177.051,-202.7112 168.5313,-196.4133 171.264,-206.6496 177.051,-202.7112\"/>\n",
       "</g>\n",
       "<!-- ensemble -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>ensemble</title>\n",
       "<polygon fill=\"#d3d3d3\" stroke=\"#d3d3d3\" points=\"311.5,-196 226.5,-196 226.5,-160 311.5,-160 311.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"269\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">ensemble</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;ensemble -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>random&#45;abtest&#45;&gt;ensemble</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M221.1499,-231.8314C228.6717,-223.3694 237.7565,-213.1489 245.9804,-203.8971\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"248.6049,-206.2127 252.6327,-196.4133 243.373,-201.5621 248.6049,-206.2127\"/>\n",
       "</g>\n",
       "<!-- transformerendpoint -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>transformerendpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"53\" cy=\"-106\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"53\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- transformer&#45;&gt;transformerendpoint -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>transformer&#45;&gt;transformerendpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M130.0088,-159.8314C115.6611,-149.802 97.7797,-137.3023 82.8599,-126.873\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"84.6681,-123.8666 74.4668,-121.0059 80.6576,-129.6039 84.6681,-123.8666\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"203.5,-124 108.5,-124 108.5,-88 203.5,-88 203.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"156\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- transformer&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>transformer&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M156,-159.8314C156,-152.131 156,-142.9743 156,-134.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"159.5001,-134.4132 156,-124.4133 152.5001,-134.4133 159.5001,-134.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"156\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"156\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M156,-87.8314C156,-80.131 156,-70.9743 156,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"159.5001,-62.4132 156,-52.4133 152.5001,-62.4133 159.5001,-62.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"316.5,-124 221.5,-124 221.5,-88 316.5,-88 316.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"269\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- ensemble&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>ensemble&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M269,-159.8314C269,-152.131 269,-142.9743 269,-134.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"272.5001,-134.4132 269,-124.4133 265.5001,-134.4133 272.5001,-134.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>classifier&#45;3</title>\n",
       "<polygon fill=\"none\" stroke=\"#000000\" points=\"429.5,-124 334.5,-124 334.5,-88 429.5,-88 429.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"382\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">classifier&#45;3</text>\n",
       "</g>\n",
       "<!-- ensemble&#45;&gt;classifier&#45;3 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>ensemble&#45;&gt;classifier&#45;3</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M297.5147,-159.8314C311.8342,-150.7074 329.3624,-139.539 344.7338,-129.7449\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"346.9417,-132.4882 353.4945,-124.1628 343.1801,-126.5847 346.9417,-132.4882\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"269\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"269\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M269,-87.8314C269,-80.131 269,-70.9743 269,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"272.5001,-62.4132 269,-52.4133 265.5001,-62.4133 272.5001,-62.4132\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3endpoint -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>classifier&#45;3endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"#000000\" cx=\"382\" cy=\"-34\" rx=\"37.0935\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"382\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;3&#45;&gt;classifier&#45;3endpoint -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>classifier&#45;3&#45;&gt;classifier&#45;3endpoint</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M382,-87.8314C382,-80.131 382,-70.9743 382,-62.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"385.5001,-62.4132 382,-52.4133 378.5001,-62.4133 385.5001,-62.4132\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f4c16720400>"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/complex_graph.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this final example we will deploy a complex graph with all of the components that have been used so far."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-deployment created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/complex_graph.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-complex-complex-41de5b8 replicas:1 replicasAvailable:1] map[name:test-deployment-complex-complex-df66c5c replicas:1 replicasAvailable:1] map[name:test-deployment-complex-complex-918397a replicas:1 replicasAvailable:1] map[replicas:1 replicasAvailable:1 name:test-deployment-complex-complex-caf37ce] map[name:test-deployment-complex-complex-61cb28d replicas:1 replicasAvailable:1]] state:Available]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":42753,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"status\": {\n",
      "    \"code\": 0,\n",
      "    \"info\": \"\",\n",
      "    \"reason\": \"\",\n",
      "    \"status\": \"SUCCESS\"\n",
      "  },\n",
      "  \"meta\": {\n",
      "    \"puid\": \"2m8ql6jk3tk453in8vmd70tdbq\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": 0.6441668056884616\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1,\n",
      "      \"ensemble\": -1,\n",
      "      \"random-abtest\": 1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"outlier-detector\": \"seldonio/mock_outlier_detector:1.0\",\n",
      "      \"ensemble\": \"\",\n",
      "      \"classifier-3\": \"seldonio/mock_classifier:1.0\",\n",
      "      \"classifier-2\": \"seldonio/mock_classifier:1.0\",\n",
      "      \"random-abtest\": \"\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.09448487573962702]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"5a69aa78-b5da-4f56-b9d4-aee99a3738d1\",\"token_type\":\"bearer\",\"expires_in\":42750,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"995vcb8j9l3drslf4iun42kcmm\"\n",
       "  tags {\n",
       "    key: \"outlierScore\"\n",
       "    value {\n",
       "      number_value: 0.2801908287422691\n",
       "    }\n",
       "  }\n",
       "  routing {\n",
       "    key: \"outlier-detector\"\n",
       "    value: -1\n",
       "  }\n",
       "  routing {\n",
       "    key: \"random-abtest\"\n",
       "  }\n",
       "  routing {\n",
       "    key: \"transformer\"\n",
       "    value: -1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"outlier-detector\"\n",
       "    value: \"seldonio/mock_outlier_detector:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"random-abtest\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"transformer\"\n",
       "    value: \"seldonio/mock_transformer:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.08528270048867242\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",None,API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/complex_graph.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tear Down"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"seldon-core\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete seldon-core --purge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"seldon-core-crd\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete seldon-core-crd --purge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
